5 research outputs found
FPO++: Efficient Encoding and Rendering of Dynamic Neural Radiance Fields by Analyzing and Enhancing Fourier PlenOctrees
Fourier PlenOctrees have shown to be an efficient representation for
real-time rendering of dynamic Neural Radiance Fields (NeRF). Despite its many
advantages, this method suffers from artifacts introduced by the involved
compression when combining it with recent state-of-the-art techniques for
training the static per-frame NeRF models. In this paper, we perform an
in-depth analysis of these artifacts and leverage the resulting insights to
propose an improved representation. In particular, we present a novel density
encoding that adapts the Fourier-based compression to the characteristics of
the transfer function used by the underlying volume rendering procedure and
leads to a substantial reduction of artifacts in the dynamic model.
Furthermore, we show an augmentation of the training data that relaxes the
periodicity assumption of the compression. We demonstrate the effectiveness of
our enhanced Fourier PlenOctrees in the scope of quantitative and qualitative
evaluations on synthetic and real-world scenes
Efficient 3D Reconstruction, Streaming and Visualization of Static and Dynamic Scene Parts for Multi-client Live-telepresence in Large-scale Environments
Despite the impressive progress of telepresence systems for room-scale scenes
with static and dynamic scene entities, expanding their capabilities to
scenarios with larger dynamic environments beyond a fixed size of a few
square-meters remains challenging.
In this paper, we aim at sharing 3D live-telepresence experiences in
large-scale environments beyond room scale with both static and dynamic scene
entities at practical bandwidth requirements only based on light-weight scene
capture with a single moving consumer-grade RGB-D camera. To this end, we
present a system which is built upon a novel hybrid volumetric scene
representation in terms of the combination of a voxel-based scene
representation for the static contents, that not only stores the reconstructed
surface geometry but also contains information about the object semantics as
well as their accumulated dynamic movement over time, and a point-cloud-based
representation for dynamic scene parts, where the respective separation from
static parts is achieved based on semantic and instance information extracted
for the input frames. With an independent yet simultaneous streaming of both
static and dynamic content, where we seamlessly integrate potentially moving
but currently static scene entities in the static model until they are becoming
dynamic again, as well as the fusion of static and dynamic data at the remote
client, our system is able to achieve VR-based live-telepresence at close to
real-time rates. Our evaluation demonstrates the potential of our novel
approach in terms of visual quality, performance, and ablation studies
regarding involved design choices
Westwater Canyon - La Sal [mountain] range from Colo. River below Cisco, Utah.
Photo of a scene along the Colorado River in Westwater Canyon near Cisco, Grand County, Utah. Taken during Harold H. Leich's river voyage down the Colorado River in 193